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ABSTRACT 

This paper examines two kinds of student engagement: 
"procedural," which concerns classroom rules and regulations; and 
"substantive," which involves sustai.^ed commitment to the content and 
issues of academic study. It describes the manifestations of these 
two forms of engagement, explains how they relate differently to 
student outcomes, and offers some empirical propositions using data 
collected on literature instruction, collected during 1987-88 from 58 
eighth-grade English classes (N=l,041 students). The results provide 
support for ';:he following three hypotheses: (l) disengagement 
adversely affects achieveinent; (2) procedural engagement has an 
attenuated relationship to achievement because its observable 
indicators conflate procedural and substantive engagement; and (3) 
substantive engagement has a strong, positive effect on achievement. 
Features of substantively engaged instruction include authentic 
questions or questions that have no prespe'^tfied answers; uptake or 
the incorporation of previous answers into subsequent questions; and 
high-level teacher evaluation or teacher certification and 
incorporation of student responses in subsequent discussion. Each of 
these features is noteworthy because it involves reciprocal 
interaction and negotiation between students and teachers, which is 
said to be the hallmark of substantive engagement. (Author/TJH) 



* Reproductions supplied by EDRS are the best that can be made * 

from the original document. * 

*'»t*ie^**^fe^****fs**********fci:****fc********$,**fi**ie*******ic************f:*** 



o 

CO 

IH 

CO 



U.S. DEPARTMENT OF EDUCATION 
Office ot Educational Research and Improvoment 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 



SThis document ha* been reproduced as 
received from the person or orcanization 
OTigiriating il 
O Minor changes have been made to improve 
reproduction Quality 

• Points of view or opinions Slated tntfits docu- 
ment do not recessariiy represent offtoal 
OERf position or pobcy 



INSTRUCTIONAL DISCOURSE AND STUDENT ENGAGEMENT 



Martin Nystrand and Adam Gamoran 



Departments of English and Sociology 
and 

The National Center on Effective Secondary Schools 
The Wisconsin Center for Education Research 
The Univciaity of Wisconsin-Madison 



June 1989 



Submitted for publication to the Harvard Educational Review 



ERIC 



The authors are grateful for insightful comments on an earlier draft of this 
paper by Fred Newmann^ for the research assistance of Mark Berends, Dae- 
dong Hahn, and Joha Knapp, and for the contributions of the teachers and 
students who participated in the study reported in this paper. This paper was 
prepared at the National Center on Effective Secondary Schools, Wisconsin 
Center for Education Research, School of Education, University of Wisconsin- 
Madison, which is supported in part by a grant from the Office of Educational 
Research and Improvement (Grant No. G-008690007-89). Any opinions, findings, 
and conclusions or recommendations expressed in this publication are those of 
the authors and do not necessarily reflect the views of this agency or tiie U. S. 
Department of Education. 



BEST COPY AVAILABLE 



Abstract 

This paper examines two kinds of student engagement: "procedural," which 
concerns classroom rules and regulations, and "substantive," which involves 
sustained commitment to the content and issues of academic study. It describes 
the manifestations of these two forms of engagement, explains how they relate 
differently to student outcomes, and offers some empirical propositions using 
data on literature instruction from 58 eighth-grade English classes. The results 
provide support for three hypotheses: (a) Disengagement adversely affects 
achievement; (b) Procedural engagement has an attenuated relationship to 
achievement because its observable indicators conflate procedural and substantive 
engagement; and (c) Substantive engagement has a strong, positive effect on 
achievement. Features of substantively engaging instruction include authentic 
questions, or questions which have no prespecified answers; uptake, or the 
incorporation of previous answers into subsequent questions; and high-level 
teacher evaluation, or teacher certification and incorporation of student 
responses in subsequent discussion. Each of these is noteworthy because they all 
involve reciprocal interaction and negotiation between students and teachers, 
which is said to be the hallmark of substantive engagement. 
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INSTRUCTIONAL DISCOURSE AND STUDENT ENGAGEMENT 
American secondary school teachers face few problems as vexing as that of 
motivating students to vork hard at their studies. Probably every teacher has 
had the experience of preparing what seemed to be an exciting, challenging 
lesson, only to see it fail to raise an eyebrow among students who at best go 
along far enough to obtain whatever grade they consider acceptable. The 
problem of creating and sustaining student engagement in schoolwork is an 
important one for at least two reasons. First, when students are engaged, 
teachers find their work more rewarding (Lortie, 1975). Second, and most 
central for this paper, engaged students are more likely to learn the knowledge 
and skills that schools have to offer. 

One difficulty in examining these propositions is that engagement is a 
difficult concept to measure. In this paper, we sugge^^t that there are at least 
two forms of student engagement, which we term "procedural" and "suDstantive." 
The first reflects an accommodation to classroom rules and regulations, whereas 
the second involves sustained commitment to the content and issues of academic 
study. The tasks of this paper are to describe the manifestations of these 
different forms of engagement, to explain how they relate differently to student 
outcomes, and to examine some empirical propositions using data on classroom 
instruction in 58 eighth-grade English classes. 
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Procedural versus Substantive Engagement 
Typically, few students are actively offtask, or disengaged (Gocdlad, 1984). 
Rather, most students are at least engaged in the procedures of their school 
tasks: they mainly pay attention; they do not distract from the business of the 
classroom; they do their v/ork; they sometimes ask questions, typically about 
what they are to do (e.g., how long their papers have to be, whether or not 
they have to learn all the terms listed at the end of a chapter, etc.); and they 
do their homework and assignments m a timely and acceptable manner. In short, 
they competently go through the motions of school. Occasionally they become 
genuinely engaged in academic problems and issues. But for most students, this 
kind of engagement is rare. In Bloome and Argumedo's (1983) terminology, the 
main kind of student engagement with school is one of procedural display; we 
call it procedural engagement. 

Academic achievement, however, requires more than competence in school 
procedures. More to the point, it requires sustained commitment to and engage- 
ment in the content of schooling, i.e., the problems and issues of academic study. 
In contrast to procedural enfgagement, which lasts only as long as the task itself, 
wc call this kind of student activity substantive engagement. Significant 
academic achievement is not possible without sustained, substantive engagement, 
which transcends procedural engagement. 

This distinction between procedural and substantive engagement helps us see 
that student engagement depends not only on students' involvement with their 
schoolwork but also on the quality of the schoolwork they invest themselves in. 
On the one hand, student engagement depends on students' psychological 
investment in class activities, and fully played out, this investment will lead to 
mastery. But *if students are required to invest themselves in little more than 
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filling in blanks, taking multiple-choice tests, and reciting and recalling super- 
ficially treated content, or, if, in English, they are taught only the formal 
requirements of English prose (usage, mechanics, the format of a five-paragraph 
theme, etc.) without any attention to content, then their engagement and 
mastery will by definition be limited to a set of procedures. Schoolwork and 
class activities will foster substantive student engagement only if these activities 
require more than a mastery of procedures. 

How can student engagement be recognized? What does an engaged student 
look like? And what empirical distinctions can be made between procedural and 
substantive engagements? There are no easy answers to these questions. 
Certainly all engaged students do not manifest their engagement in the same 
way. In class, for example, procedural engagement is characterized by normal, 
unproblcmatic, but otherwise undistinguished behavior; hence, proceduralJy 
engaged students are less likely to be offtask than disengaged students. By 
contrast, substantively engaged students may well ask more questions than other 
students, especially about the content of study, and not just about how many 
words they need to write, or whether they may use pencil instead of pen. But 
clearly these behavioral manifestations are incomplete and inadequate as measures 
of student engagement. And often they do not distinguish between students who 
are procedurally engaged and thos^ who are substantively engaged. Substantive 
student engagement may manifest itself as a twinkle in the eye, or it may not; 
substantive student engagement may manifest itself as rapt attention for a long 
period of time, or it may not; substantive student engagement may for many 
students manifest itself only years after they leave school, or it may not. Many 
instances of substantive engagement are no more visible than thought itself. 
Conceivably some examples of substantive engagement may even manifest 
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themselves in terms of significant offtask behavior, as in the example of students 
so completely absorbed by a scientific project that they do no English in English 
and no social studies in social studies. Clearly, the manifestations of student 
engagement are sundry, ambiguous, and elusive. Often there simply are no clear 
behavioral manifestations of engagement, certainly nc uniform manifestations. 

Quality of Instructional Discourse and Substantive Student Engagement 
We have chosen to investigate the quality of instructional discourse as a 
source of data on student engagement. Mehan (1979) showed that classroom 
question-answer episodes typically have three parts: an initiation (the teacher^s 
question), a response (a student's answer), and an evaluation (the teacher's 
response to the student*s answer). This three-part structure characterizes the 
normal procedures of classroom recitation, and students who regularly interact 
with teachers in these sequences may be said to be procedurally engaged. 
Typically, teachers' evaluations of student answers are a perfunctory "Right" or 
"Wrong," a "Good" or an "Okay," sometimes merely a nod, sometimes nothing. In 
variations on this sequence of initiation-response- evaluation, however, some 
teachers respond more substantially. For example, a teacher might respond to a 
student answer by saying, "Good point," and then asking a followup question. 
When teachers do this, they work students* answers into the fabric of an 
unfolding exchange, and as these answers modify the topic or affect ihc course 
of discussion in some way, these teachers certify these contributions and 
modifications. In our project, we call such evaluations high-level evaluation. 

In recitation, teachers typically ask a series of preplanned questions to test 
students* knowledge. The topics ccvevcd ar:^ entirely the teachers*, and rarely 
does the teacher interact with the substance of students* answers except to 



evaluate them. This is why the engagement of students who participate in 
recitation is rarely more than procedural. By contrast, high-level evaluation is 
substantively engaging because the teacher acknowledges and builds on the 
substance of what the student says. 

Another way that teachers substantially engage their students in question- 
and-answer is by not prespec^fying the answers to their questions. For example, 
they ask open-ended questions, or they ask questions to which they really don*t 
know the answers. Like high-level evaluation, these questions, \vhich we call 
authentic questions to distinguish them from test questions, signal to students 
the teachers* interest in what students think and not just whether they know 
what someone else thinks or has said. 

Yet another way that teachers substantially engage their students in 
question-and-cnswer is by following up on student answers by incorporating 
these answers into subsequent questions, a process that Cazden (1988), Collins 
(1982, 1986), and others call uptake. Here is an example: 

Teacher: What is a lobbyist? 

Student : Someone who represents someone else. 

Teacher: "Represents" for what purpose? 
In this exchange the teacher*s second question is an example of uptake because 
it incorporates and, in this case, actually quotes part of the student^s response 
("represents"). High-quality instructional discourse frequently manifests uptake 
because, like a chentic questions, it accommodates input from students. 

Taken together, high-level evaluation, authentic questions, and uptake are 
aspects of classroom discourse in which student-teacher exchanges unfold not 
simply according to the teacher*s preplanned agend*- of questions, but rather 
where teachers and students work in terms of each other, and where, as a 
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result, the course of classroom talk depends on what both teachers and students 
bring to the instructional encounter. When teachers ask authentic questions, 
they open the floor to what students have to say; when they engage in uptake, 
they build on what students have said; and when their evaluation of student 
responses is high, they certify new turns in the discussion occasioned by student 
answers. These aspects of classroom discourse, which lend thematic coherence to 
the talk by interweaving discussion topics across teacher-student turns, serve to 
sustain student-initiated ideas and responses. 

Such interleaving of discourse topics from one speaker to the next, of 
course, is common in — indeed is the very structure of - conversation as the 
conversants typically pick up on each other's comments. Substantively engaging 
classroom talk is hence a lot like conversation. In neither conversation nor high 
quality instructional discourse can the substance be specified in advance by any 
of the conversants. Just as the substance and conduct of talk are negotiated in 
the process of conversing (Sacks, Schegloff, and Jefferson, 1974), substantively 
engaging instruction is created through a process of negotiation between teachers 
and students (Nystrand and Gamoran, 1988). 

The extent to which class discourse resembles conversation is in fact an 
excellent criterion for judging both the instructional quality of classroom 
discourse and the extent of substantive student engagement. By this, we do not 
mean to suggest that instruction should be given over to idle chatter, but rather 
that students are most likely to be substantively engaged when the treatment of 
subject matter aHows for extensive interaction. Hence, substantive student 
engagement is often high in small-group work and discussion; it is much less 
likely in lecture; and it generally exists in question-and-answer only to the 
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extent that questions are authentic, teacher evaluations arc high-level, and 
uptake is present. 

In recitation, most questions are asked by the teacher, get a response and 
a low-level evaluation, are test questions involving no uptake, and elicit the 
cognitive level of a report. By contrast, in collaborative work among peers in 
small groups, all the questions are asked by students, are authentic and typically 
exhibit uptake in just the way that discussion and conversation are typically high 
in these values. In both recitation and small-group work, then, uptake, authen- 
ticity, and level of evaluation are useful criteria for judging the quality of 
instructional discourse and, consequently, student engagement. We argue that 
question-answer exchanges between teachers and students that are characterized 
by authentic questions, uptake, and high-level evaluation are substantively 
engaging. Collrhorative, small-group work and discussion which also exhibit 
these characteristics are also substantively engaging. By contrast, question- 
answer exchanges that are low in these terms will rarely be more than procedur- 
ally engaging. 

By "coUac .rative work" and "discussion," of course, we are referring to ;real 
small-group work and real discussions where students have some input into and 
control over the discourse; we do not include here small group time that is nsed 
to complete worksheets, an activity which might more properly be called 
"collaborative seatwork." 

Procedural engagement is more or less obvious, we have noted, from the 
direct observation of individual students: they do their work, are not disruptive, 
etc. But substantive engagement is more complicated and often cannot be 
ascertained by scrutinizing the Dchavior of individual students alone. Substantive 
engagement is usually obvious in student-teacher and peer interactior*^ ^vhere the 
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convcrsants clearly work in terms of each other, e.g.» where the teacher picks 
up on the substance of a student*s response and where, consequently, the topic 
is sustained across conversation turns. To put this in other terms, substantive 
engagement requires a high degree of reciprocity between convcrsants (cf. 
Nystrand, 1986; Nystrand and Gamoran, 1988). As Cummins (1986) notes, 
rcciproca3-intcraction instruction requires "a genuine dialogue between student 
and teacher in both oral and written modalities, guidance and facilitation rather 
than control of student learning by the teacher, and the encouragement of 
student/student talk in a collaborative learning context" (Cummins, 1986, p. 28). 
Cummins goes on to note the chief features of such instruction: 

This model emphasizes the development of higher level cognitive skills 
rather than just factual recall, and meaningful language use by 
students rather than the correction of surface forms. Language use 
and development arc consciously integrated with all curricular content 
rather than taught as isolated subjects, and tasks are presented to 
students in ways that generate intrinsic rather than extrinsic motiva- 
tion (Cummins, 1986, p. 28\ 
Whereas procedural engagement is possible when teachers dominate the agenda of 
classroom learning, substauvlvc engagement is possible only in instructional 
arrangements where students as well as teachers have input into the business of 
learning. 

Like classroom discourse, writing and reading can be either procedurally or 
substantively engaging or both. In our research, therefore, we have examined 
not only class discussion^ but ali^o writing and reading in these ter^ns - the 
universe of instructional discourse,, that is, as it affects the learning of students 
in our study. 
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One type of writing componly taught in high school that is far more 
procedurally than substantively engaging is the 5-paragraph theme, an expository 
formai: consisting of (a) an introductory paragraph stating a thesis, (b) a body of 
three paragraphs, each one stating a different argument, followed by (c)^a 
concluding paragraph. The stipulation of this format for three main points is 
categorical and independent of the writer^s subjeov and argument. Hence, 
students need not figure out if their arguments actually have more or fewer then 
three main points; one of their first tasks is to devise three main points. There 
is no authenticity in these tasks sinct teachers prcspecify the type of response, 
i.e., 5-paragraphs, that they expect. Moreover, if teachers evaluate students* 5- 
paragraph themes muinly in terms of adequacy of form, level of evaluation will 
be low. In short, the conception of an essay as a 5-paragraph form reduccs- 
somc would say "triviaHze^^ - exposition to a procedure, as do all types of 
writing instruction in which content, substance, and writer purpose arc subju- 
gated to form and procedure. Br^tton ct al , (1975) call these writing tasks, 
which are endemic to schools, dummy runs. 

By contrast, some English teachers ask students to write position papers in 
which they must further articulate the positions they take in lively class 
discussions. In cases such as these, the discussion serves as pr^writing, and, 
because the students write about an argument they C2.re about and therefore 
havQ spmethipR t,P 9^Y> the writing task is often substantively engaging, not to 
mention intrinsically rewarding and interesting. Authenticity will be high m 
such cases since the teacher does not prespceify the content of papers, but level 
of evaluation will depend, of course, on whether or not the teacher responds to 
the paper by picking up on the content of the paper in his or her evaluation. 
Applebee and Langer (Applcbec and Langer 1983; Langer and Applebee, 1984, 
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1986) have argued that such writing tasks promote student "ownership" because 
they afford students considerable flexibility concerning the content they cover 
and the views they express. 

This example manifests another characteristic of high quality instructional 
discourse: the frequency that one instance of discourse adjoins another. 
Examples include prewriting that leads to writing, writing that leads to rewrit- 
ing, writing that leads to reading, reading that leads to writing, discussion that 
leads to either, and writing and reading that lead to discussion. Uptake is also 
an example in the sense that teachers* comments pick up on and are therefore 
contiguous with a student*s contribution. In our study, we include all such 
examples of discourse enriched by other disco'^rse under the heading of discourse 
contiguity. 

Another type of writing instruction that is often substantively engaging 
involves peer conferencing, a method requiring students to review and discuss 
their writing with each other in groups of four or five. In peer conferencing, 
teachers give students no checklists to use in monitoring their discussions. 
Rather, they encourage groups to focus on the clarity and adequacy of what 
each writer has to say, i.e., on the writer*s purpose and the substance of the 
paper, and instruct them to avoid checking spelling, punctuation, and usage. In 
these peer groups, the talk is authentic since students typically don*t quiz cach 
other but exchange onlv that information they actually need to know. Uptake is 
also high so long as the conversants listen and respond appropriately to each 
other. 

Journal keeping is yet another type of writing that is substantially engaging 
for many students. In keeping journals, students get to write about topics that 
interest them, which is to say, teachers prespecify none of the content; in our 
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if.' IS, the discourse of journals is highly authentic. Journal entries are often 
highly expressive and loosely organized, almost always informal: it is okay for 
students to be tentative and to digress in ways that would be inappropriate in 
more formal assignments and tests. As a result, teaehers usually do not grade 
journals though they eount them for eredit if students regularly make entries. 
Rather than marking and grading them, teaehers respond in the margins to the 
eont^nt of the entries; eommenis like "Very interesting! Tve thought that too** or 
"Have you ever stopped to think . not uncommon. The cumulative 

effect of journal entries and teacher responses is that of a written dialogue or 
conversation; indeed keeping journals is sometimes called dialogue-journal 
communication (cf. Staton, Shuy, Kreeft Peyton, and Reed, 1988). Students and 
teachers take turns speaking just as conversants do. As in conversation, 
teachers typicsily pick up on and comment on the substance of the entries, and 
by doing so, they sustain the dialogue. In our terms, uptake and level of 
evaluation are both high. 

Since journal keeping usually allows students to write about just those 
things that interest them, one may reasonably ask whether journal keeping has 
any real payoff in academic achievement or writing development. Is journal 
keeping really good practice for writing themes and tests? How much "transfer" 
can there be in such tasks? Will students write better simply because they write 
frequently? These questions about "practice" and "transfer" misconceive the 
benefits of both writing devclooment and journal keeping. For while journal 
entries may be poor models of academic prose, the regular activity of keeping a 
journal is valuable for helping students to get to know their teacher as a reader 
who is interested in what they have to say and how they think, that is, as 
someone who is open to their ideas and responds, not just someone who assigns 
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homework and evaluates their performc»nce. After they have written journal 
entries for a while and gotten to know this interested, receptive reader, they 
wiJ! presumably be more willing to take the kinds of accessary risks required by 
higher order thinking and more formal v/ork. 

Reading is authentic to the extent that it addresses questions that students 
deem are important, teaching them new things that they value, and also to the 
extent that teachers help students relate their readings to their own experiences. 
Contiguity of reading will be high to the extent that students discuss and write 
about their readings — in other words, to the extent that reading relates to talk 
and writing. 

Table 1 shows the potential of selected instructional activities for substan- 
tive student engagement. 



Table 1 about here 



Hypotheses; Concerning Procedural Engagement. Substantive Engagement. 

and Academic Achievement 

Student engagement is a cognitive phenomenon having to do with the extent 
to which students arc mentally involved with the issues and problems of academ- 
ic study. And so it is appropiiatc to describe student engagement in terms of 
sustained mental concentration, focus, and habits of thoughtfulncss (Ncwmann, 
Onosko, & Stevenson, 1988). But like most aspects of cognition, student 
engagement has a social foundation. Substantive student engagement involves 
more than individual students: more precisely, it involves the interaction of 
students and teachers. This requirement for interaction clearly underlies the 
social nature of instruction. 
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In classes where students are substantively engaged, their involvement with 
academic content is likely to be manifested not in one or two indicators, but in 
a broad pattern of instructional discourse. Teachers who encourage substantive 
engagement, and students who become so engaged, may well be immersed in a 
dialogue that spans a variety of classroom activities. We would expect, for 
example, that classes devoting significant amounts of time to discussion and 
peer-group work, probably exhibit authenticity, contiguity, and a high level of 
teacher evaluation in their reading, writing, and question-answer activities. 

We would further expect to find distinct patterns of influences associated 
with disengagement, procedural engagement, and substantive engagement, each 
considered separately, on academic achievement. First, we would expect to find 
a negative relation between achievement and disengagement. Students who are 
not even procedurally engaged in their schoolwork — those who misbehave in 
class, who fail to do homework, and so on — can hardly be expected to learn 
anything. Second, we would expect the effects of procedural engagement, while 
perhaps positive, to be weak. This is because readily observable indicators of 
procedural engagement — such as doing homework, asking questions in class, 
and so on — conflate procedural and substantive engagement. By contrast, we 
expect to find unequivocal, positive effects of substantive engagement on student 
learning of subject matter. Such engagement, as we describe below, can be 
measured by examining the frequency of substantively engaging activities such as 
discussion, small-group work, authentic questions, and the like. 

In studies of peer conferencing, for example, Nystrand (Nystrand, 1986, 
chapter 8, Nystrand and Brandt, 1989) showed that students who participated in 
peer conferencing learned to write better expository writing than others who 
wrote only for the teacher. Those engaged in peer conferencing increasingly 
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viewed their readers as collaborators in a process of communication and treated 
revision as a matter of reconceptualization, whereas the group writing for the 
teacher increasingly viewed their readers as judges and treated revision as a 
matter of editing. This study provides empirical support for the notion that 
instruction that is collaborative and substantively engaging, involving give-and- 
take on both sides, effectively promotes learning. 

Data 

Assessing these propositions requires fine-grained quantitative data on what 
teachers and students do in classes. During 1987-88, we collected data from 58 
eighth-grade English classes in 16 Midwestern schools.^ (Data were also 
collected from social studies classes, but they are not used in the present paper.) 
The schools selected differ in the compositions of their communities: three are 
rural and all-white; four are suburban, mostly white, and mostly upper-middle- 
class; and nine are urban schools that vary in their socioeconomic and ethnic 
make-ups. Ten of the schools are public junior highs or middle schools, and the 
remaining six are Catholic K-8 schools. In the Catholic schools, and in the rural 
and suburban public schools, all the eighth grade English classes participated. In 
the public urban schools, which tended to be larger, four classes from each took 
part. Of 1171 students in these classes at the beginning of the year, 1041 
participated in the study throughout the year, for a participation rate of 89%. 
(About 8% of the students were absent or declined to take part, and another 3% 
were lost through attrition.) 

^ What we arc calling "English classes" were variously defined by the 
schools as English, reading, communications, literature, etc. In each school, we 
studied the eighth grade classes in which reading constituted the primary 
activity. 
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Data sources included student tests and questionnaires, teacher question- 
naires, and classroom observations. Each class was visited four times by 
observers trained to evaluate the features of classroom instruction described 
below Essentially, the classroom observation scheme focused on two tasks: 
noting the time spent in different types of activities, such as lecture, seatwork, 
discussion, and so on; and recording and coding the questions asked in class 
according to their authenticity, uptake, level of evaluation, and other aspects of 
instructional discourse. Data for each class were averaged across the four 
observations. 

Indicators of Disengagement and of Procedural and Substantive engagement 

Indicators of disengagement included student-reported frequency of failing 
to turn in written work and to do reading assignments; the percentage of 
students clearly offtask during question-answer periods as seen by an observer; 
and the proportion of questions asked by teachers that received no response at 
all from students. We expected these variables to be negatively related to 
learning. 

Indicators of procedural engagement included the number of hours per week 
students reported spending on homework for English class; student reports of the 
frequency with which they asked questions about what they were supposed to 
learn; and the proportion of students who were observed actively participating 
during question-answer sessions. (The latter figure, along with the proportion 
offtask, does not sum to 100% of each class because it does not include students 
sitting passively.) These variables may or may not contribute to learning 
depending on the extent of substantive engagement. 
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To measure substantive engagement, we operationalized concepts such as 
authenticity, uptake, and level of evaluation. During class observations, each 
question asked by teachers and students was written down and coded. (In case 
of rapid question-answer sequences, questions were recorded but not coded until 
after the observation.) The variable "authenticity of tv .cher questions" was 
constructed as the sum of authentic and quasi-authentic questions (with the 
latter weighted as one-half the former) divided by the total number of teacher 
questions.2 "Uptake" indicates the proportion of questions that involved uptake, 
and "High-level evaluation" reflects the number of high-level responses divided 
by the number of teacher questions. 

From teacher questionnaires, we also collected information on the quality of 
discourse embodied in reading and writing assignments. Authenticity of writing 
was indicated by the frequency with which students wrote in response to 
questions that had no prespecified answers. Authenticity of reading reflects 
authentic treatment of reading assignments, indicated by the extent to which 
teachers asked students to relate their reading to their own experiences, and by 
asking students to give their opinions about the readings. High-level evaluation 
of writing was assessed on a standardized scale through questions about how 
teachers respond to written work and how they communicate their evaluations to 
students. Contiguity of writing indicates the frequency with which students 
write about what they have read, and discuss topics prior to writing. Similarly, 



^ An authentic question is a question whose answer is not prespecified by 
the teacher. Examples include, "What did you think of the story?" or "Why do 
you say that?" By contrast, quasi-authentic questions are questions that allow 
some degree of student control over the flow of discussion. An example is, "Can 
you name one of the three causes we reviewed yesterday for the American 
Revolution?" This kind of question allov/s students some latitude; in this case 
the student is free to cite any one of the three causes. 
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contiguity of reading reflects how often classes discussed assigned readings and 
related readings to other readings. 

Finally, discussion time and small-group time indicate the average daily time 
each class spent in these two activities. A distinction was made between 
question-answer time and discussion time, with the latter reflecting only the free 
exchange of comments, often uninterrupted by the teacher. 
Indicators of Stude nt Background and Achievement 

Our ultimate goal was to document the connection between engagement and 
achievement. Consequently our study required measures of student learning, as 
well as indicators of student background and prior achievement, the purpose of 
the latter to control for external influences on learning and for students* prior 
skills. 

Student learning was measured with a test of literature achievement 
administered in the spring of 1988. The test posed a series of questions ranging 
from simple recall to in-depth understanding and synthesis concerning five pieces 
of fiction vhat students had read during the year. The five readings were 
selected to represent a list of all fiction covered in class during the year. Thus, 
the questions on the test were of the same type in all 58 classes, but the stories 
they concerned differed from class to class. For example, all students were 
asked to describe the conflicts in five readings, but the list of readings varied, 
depending on what they were supposed to have read. These tests were holis- 
tically scored on dimensions such as extent of recall, depth of understanding, 
identification of characters* motivations, and so on. Each test was scored by 
two raters, and the marks were averaged. When the marks differed by more 
than one point on any given item, the items received an additional two readings. 
Interrater agreement was calculated as a correlation of .90. 
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To control for students* prior capacity writing and reading abilities, two 
tests were administered in the fall of 1987. The first test elicited a writing 
sample from each student. It was scored by two readers, whose marks were 
averaged, on two dimensions: level of abstraction, based on Britton ct al. 's (1975) 
categories of transactional-informative prose; and coherence of argumentation, 
based on the 1979/1984 NAEP criteria for informative writing (in Applebee, 
Langer, & Mullis, 1985). Interrater reliability of scoring this test was .70. The 
second, a test of reading power taken f:om the National Assessment of Educa- 
tional Progress (NAEP), posed multiple-choice questions about a series of poems 
and narrative passages. This test al5.o included a brief writing sample, which 
was scored using NAEP (1979) criteria for the identification and substantiation of 
personal emotions and feelings elicited by a short story. Scores obtained from 
two readers of this passage, which were averaged and added to the multiple- 
choice scores, correlated at .86. Looked at longitudinally, then, our tests 
address the following question: Given an initial level of reading and writing 
ability, in what ways do students differ in their attainment of mastery over their 
literature curricula? 

Further controls included data on student background characteristics. 
Student questionnaires yielded information on students* race, ethnicity, sex, and 
socioeconomic status (SES). The latter was constructed as an unweighted 
additive composite of student reports of mother*s education, father*s education, 
the higher of mother*s or father*s occupation on an updated Duncan SEl scale 
(Stevens and Cho, 1982), and possession of a list of home resources. Because a 
few schools placed seventh graders in soiiie of the eighth-grade classes, we also 
included a control variable for each student*s grade level. 
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Results 

Our general argument can best be supported by showing that substanti^'cly 
engaging instruction tends to occur in patterned ways across classrooms, and by 
supporting our propositions about the importance of different forms of engage- 
ment for student achievement Before turning to these analyses, however, it is 
useful to examine the amount of procedural and substantive engagement found in 
the classes in our sample. This information is provided in Table 2. 



Table 2 about here 



In most ways, the 58 classes appear to be characterized by a high degree 
of procedural engagement but little substantive engagement. Few students are 
offtask; most do their homework though they spend less than an hour a week on 
it; they ask questions onc3 in a while; almost all of the teachers' questions elicit 
some answer; and about a quarter of the students in the classes appeared to be 
active during question-answer sessions. At the same time, there was very little 
discussion and small-group work -- an average of less than a minute per day in 
these classes. The average proportion of high-level evaluations was just 3%, and 
the average proportion of questions that involved uptake was only 11%. Stu- 
dents' writing tasks were authentic barely more than once per week, although 
the students tended to treat their readings authentically much more frequently. 
While an average of only 12% of the questions were authentic, another 22% wer». 
quasi-authentic, for an authenticity score average of 23.2%. Still, the overall 
picture appears highly consistent with earlier descriptions of secondary school 
classrooms as orderly but lifeless (Sizer, 19S4; Goodlad, 1984; Powell, Farrar, and 
Cohen, 1986). 
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Wc have suggested that classes in which the quality of discourse was high 
would spend more time on activities that inherently involve substantive engage- 
ment: small-group work and discussion. Tabic 3 provides support for this 
prediction, especially in the case of discussion. Teachers who spend more time 
in discussion were more likely to assign authentic writing tasks and to respond 
to writing at a high level, more likely to treat readings authentically, and more 
likely to use uptake and other forms of contiguity in their lessons. 

Table 3 about here 



The core of our analysis is the examination of the relations between 
procedural and substantive engagement, on the one hand, and academic achieve- 
ment, on the other. Table 4 presents regression analyses in which achievement 
on the literature test served as the dependent variable, to be explained by 
variation in conditions of background, disengagement, procedural engagement, and 
substantive engagement.^ 



Table 4 about here 



Results are displayed from four regression equatinns. In the first column, 
achievement is regressed on only the background variables. As expected, 

^ Given our expectation that high-quality instructional discourse would 
appear as a pattern of interaction in some classes, it was not surprising to learn 
that the indicators of substantive engagement covaried to a certain degree. This 
collinearity caused problems in the estimation of regression models, and wc were 
forced to delete three variables from the analyses: evaluation of oral responses, 
authenticity of writing, and contiguity of writing. Thus, although we will discuss 
the impact of specific variables in describing the results, both our strategy of 
deleting collinear variables and our conceptualization of patterns of engaging 
discourse should lead one to take more seriously the overall pattern of results 
than the contributions of individual indicators. 
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students who performed higher on the fall tests of readiny and writing main- 
tained their advantage in the spring. In addition, non-blacks, non-Hispanics, and 
eighth graders scored higher than blacks, Hispanics, and seventh graders, 
respectively. SES also contributed positively to literature achievement. The 
difference between boys and girls on the test is statistically insignificant. 

The relevance of procedural engagement appears in the second column. As 
we predicted, disengagement has a strong negative impact on achievement. 
Coefficients from these percentile variables must be read with their scales in 
mind: for example, the coefficient for offtask indicates that a 10% percent 
increase in the percent of students offtask would lead to a decline of 2.27 points 
in a studcnt*s test score. (The coefficient of .227 must be miiltiplied by 10 to 
arrive at this statement.) Similarly, an increase of 10% in student nonresponse 
to questions would drive achievement about 2 points downward. Further, for 
every 10% of writing assignments that students failed to complete, achievement is 
about three-tenths of a point lower. Thus, other things being equal, a student 
who did half his work could expect to score 1.5 points - almost a quarter of a 
standard deviation - below one who turns in all his work. 

Also in line with our expectations, the influence of procedural engagement 
is more attenuated. Spending more time on homework did lead to higher 
achievement in our results: for each additional hour per week, students gained 
just under half a point. Given the amount of time this involves when cumulated 
Over the school year, this effect appears modest. Moreover, the frequency with 
which students said they asked questions in class was unrelated to achievement. 
Finally, the observed percentage of students active during question-answer 
sequences exhibits a surprising, negative effect. Overall, these /esults support 
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our contention that procedural engagement may contribute to achievement, but 
that the relation is ar..biguous. 

The reason procedural engagement does not always lead to greater learning, 
wc argued, is that it depends on what students are engaged with. Significant 
achievement is likely to occur only to the extent that students are engaged in 
the substance of academic issues. Column 3 of Table 4 shows the importance of 
substantive engagement for literature achievement. Students scored higher on 
the achievement test in classes exhibiting more uptake, more authenticity in 
question-answer, more contiguity of reading, and more discussion time. Uptake 
and discussion time appear to exert particularly strong effects: a 10% increase in 
uptake would lead to an advantage of more than a point; and for each additional 
daily minute of discussion, achievement would be four-tenths of a point higher. 

The only anomalous finding in column 3 is the negative effect of small- 
group time. We suspect th5s occurred because, despite our conceptualization, 
what we counted as small-group time more often than not may have failed to 
include collaborative efforts among peers to solve challenging problems, but 
instead involved filling out worksheets and answering chapter questions in 
groups. 

The full model of engagement and achievement is presented in column 4. 
The results here arc very similar to what we found in columns 2 and 3: disen- 
gagement detracts considerably from achievement; procedural engagement has 
some positive effects (of homework time); and substantive engagement, as 
indicated by the quality of classroom discourse, reveals a significant tie to 
academic achievement. 

It is instructive to compare the sizes of the coefficicnis across columns. 
First* the fact that the coefficients for the engagement variables arc nearly the 
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same size in column 4 as they arc in columns 2 and 3 suggests that the iwo 
forms of engagement arc largely independent of one another, at least to the 
extent that they relate to achievement. Tl.is result is also reflected in the 
increase in the amount of variance explained by the final model ./hen compared 
to either of the preliminary models. As column 4 shows, our full model accounts 
for nearly half the variance in literature achievement. 

Second, it is important to note the decrease in the .^ize of the coefficients 
for the background variables when one compares columns 1 and 4. The fall test 
effects appear smaller, particularly the effect of writing ability. Of the other 
background variables, only SES exerts a significant effect in the final model, and 
AS coefficient is reduced in size as arc the others. This pattern indicates that 
differences between advantaged and disadvantaged groups - high and low 
achievers, those from high versus those from low SES backgrounds, and so on - 
can be attributed in part to the schooling variables that are added to the model 
in columns 2 and 3. In short, the more advantaged students in our sample had 
more profitable schooling experiences. 

Conclusions 

Student engagement poses some puzzles for both t jhers and researchers. 
On the one hand, it underscores the importance of individual student effort and 
commitment to schooling. And clearly, when we speak of student engagement, 
wc have in mind individual students, not classes. Yet despite the fact that 
student engagement refers to the cognition of individual students, we seem 
unable to detect it or adequately describe its manifestations except in relation to 
the interactions of students with their teachers or with other students. When 
w/- attempt to describe what individually engaged students do or look like, we 
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inevitably limit ourselves to describing procedural engagement: we speak 
students who appear to be paying attention, who do their work, who ask 
questions irt class, and so on. By contrast, in order tc describe substantively 
engaged students, we must turn to the general conditions of the class itself: 
when students ask questions, for example: Do their teaches follow up on these 
questions? How much latitude do students have in answering teacher questions? 
Do these questions mainly test their knowledge of what other people have 
thought and said, or do they respectfully elicit and follow up on actual thinking? 

Substantive engagement and attendant reciprocity between teachers and 
their students is important for both low- and high-ability students. For low- 
ability students, it provides essential, because intrinsic, motivation. For high- 
ability students, it is key to developing higher order thinking. More than this, 
Cummins (1986) argues, substantive engagement is critical for disengaged minority 
students who, experiencing large classes, endless remediation, and top-down 
insxruction, too often learn that what they have to say is irrelevant or wrong. 

Substantive engagement depends not just on teachers transmitting important 
knowledge and presenting good lessons. Nor does it depend just on students 
paying attention, taking in information, and doing their work. More fundamen- 
tally, it depends on what teachers and students do together and how they work 
in terms of each other; neither can do it alone. Nonetheless, teachers are key 
to creating classrooms where reciprocity is respected and possible. For example, 
teachers must carefully attune their questions and assignments to student 
interests, expectations, and abilities, which they must take seriously and obvious- 
ly respect. Teachers must be alert to the possibilities for instruction in the 
interests and questions their students bring to class, and they must be quick and 
flexible as they capitalize and follow up on these interests and questions. 
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Tabic 1. Potential of Selected Instructional Discourse Types for Substantive Student Engage- 
ment. 



Activity Type 

CLASSROOM DISCOURSE 
Recitation 

Nonrecitational 
question-and-answer 

Small-group work 
& peer conferencing 

Collaborative 
seatwork 

Lecture 

Discussion 

WRITING 

Journals & 
learning logs 

5-paragraph themes 

Position papers 

READING 



Authenticity Contiguitv/Uptake Level of Evaluation 



LOW 
VARIES 

HIGH 

NONE 

LOW 
HIGH 

HIGH 

LOW 
HIGH 
VARIES 



LOW 
VARIES 

HIGH 

NONE 

LOW 
HIGH 

HIGH 

LOW 
HIGH 
VARIES 



LOW 
VARIES 

HIGH 

LOW 

LOW 
HIGH 

HIGH 

LOW 
HIGH 
N/A 
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Tabic 2. The quality of instructional discourse: means and standard deviations. 



Variable 

DISENGAGEMENT 
Offtask in class 
Nonresponse to questions 
Reading not completed 
Writing not completed 



Mean 



4.60 pcrccr.t 
2.42 percent 
16.23 percent 
13.68 percent 



Standard 
Deviation 



6.37 
3.26 
23.43 
21.38 



Data 
Source 



Class observation 
Class observation 
Student Report 
Student Report 



PROCEDURAL ENGAGEMENT 



Active in class 
Asking questions 
Time on homework 



25.75 percent 
6.06 times/month 
.94 hours/week 



SUBSTANTIVE ENGAGEMENT 

Authenticity of questions 23.16 

High eval of oral response 3.01 

Uptake 11.41 

Discussion time .86 

Small-group time .65 

Authenticity of reading 21.65 

Authenticity of writing 5.06 

High evaluation of writing -.70 

Contiguity of reading 10.93 

Contiguity of writing 13.42 



percent 

percent 

percent 

minutes/day 

minutes/day 

times/month 

times/month 

[standardized] 

times/month 

times/month 



16.37 Class observation 
10.12 Student Report 
1.06 Student Report 



13.01 Class observation 

4.44 Class observation 

7.15 Class observation 

1.79 Class observation 

2.50 Class observation 

12.50 Teacher Report 

4.62 Teacher Report 

.50 Teacher Report 

2.68 Teacher Report 

9.05 Teacher Report 



N=58 classes for observed and teacher-reported variables, and 1088 students for 
student-reported variables, with small fluctuations due to item nonresponse. 
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Table 3. Correlation of discussion and small-group time with discourse variables (n=58 
classes). 

Discussion Small-group 

Discourse variable Time Time 

Authenticity of questions .02 -.02 

Authenticity of reading .24 .16 

Authenticity of writing^ .40 .20 

High evaluation of oral response .08 -.03 

High evaluation of writing .28 .07 

Uptake .19 25 

Contiguity of reading .23 ,08 

Contiguity of writing^ .35 J3 



^ N = 57 classes due to teacher nonrcsponse. 
^ N = 53 classes due to teacher nonresponse. 
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Table 4. Effects of disengagement, procedural engagement, and substantive engagement on 
literature achievement. Metric regression coefficients, with standard errors in parentheses. N 
a 924 students, missing values deleted listwise. 



DEPENDENT VARIABLE: SPRING LITERATURE ACHIEVEMENT 

Independent Variables Models 

Procedural Substantive Full 
Background Engagement Engagement Model 

BACKGROUND 

Sex (l=female) .438 .467 .665 .631 

(^^9) (.363) (.353) (.343) 

Race (l=black) - ^/C*'"' -1.546* -1.942** -1.185 

(.656; (.635) (.623) (.606) 

Ethnicity(l=Hisp) -1.508* -.153 -1.642** -.658 

(.641) (.622) (.602) (.590) 

SES 1.619*** 1.102*** 1.409*** 1.013*** 

(.241) (.238) (.233) (.233) 

Grade (l=eighth) 2.090*** 1.100* 1.465* .368 

(.573) (.564) (.571) (.566) 

Fall reading .390*** .299*** .363*** .298*** 

(.038) (.037) (.036) (.036) 

Fall writing .930*** .695*** .711*** .559*** 

(.148) (.142) (.140) (.135) 

DISENGAGEMENT 

Off task in class -.227*** -.177*** 

(.030) (.033) 

Reading not completed -.019 -.015 

(.010) (.010) 

Writing not completed -.031** -.025* 

(.012) (.011) 

Nonrcsponse to questions -.201*** -.228*** 

(.057) (.057) 

PROCEDURAL ENGAGEMENT 

Active in class -.032* -.063*** 

(.016) (.016) 

Asking questions -.022 -.027 

(.018) (.017) 

Time on homework .445** .398* 

(.170) (.163) 

SUBSTANTIVE ENGAGEMENT 

Authenticity of questions .035* .028 

(.015) (.015) 

Authenticity of reading ,054 .066* 

(.028) (.028) 

High evaluation of writing -.586 -.153 

(.402) (.400) 

Uptake .119*** .086** 

(.027) (.032) 

Contiguity of reading .332** .278* 

(.117) (.118) 

Discussion time .413*** .330** 

(.112) (.108) 

Small-group time -.144* -.195** 

(.071) (.069) 

R-SQUARE .322 .399 .418 .470 
* p<.05 ** p<.01 *** p<.001 
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APPENDIX 

Table A. Means and standard deviations of variables included in regression analyses. N=924 
students. 

Variable Mean Standarg Deviation 



TEST SCORES 

Spring literature 14.636 6.779 

Fall reading 21.619 5.311 

Fall v/riting 6.412 1.388 

BACKGROUND 

Sex (l=female) .508 .500 

Race (1 -black) .090 .286 

Ethnicity (iFHispanic) .102 .303 

SES .016 .825 

Grade (l=eighth) .879 .327 

DISENGAGEMENT 

Offtask in class 5.146 6.790 

Reading not completed 15.722 22.913 

Writing not completed 13.205 12.217 

Nonrcsponse to questions 2.526 3.297 

PROCEDURAL ENGAGEMENT 

Active in class 22.270 12.217 

Asking questions 5.674 9.756 

Time on homework .957 1.069 

SUBSTANTIVE EN JAGEMENT 

Authenticity of questions 23.487 13.265 

Authenticity of reading 20.235 12.219 

High evaluation of writing -.038 .484 

Uptake 11.030 7.348 

Contiguity of reading 10.658 2.752 

Discussion time .770 1.716 

Small-group time .705 2.626 
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